Efficient Unsupervised Content-Based Segmentation in Stereoscopic Video Sequences

This paper presents an e cient technique for unsupervised content-based segmentation in stereoscopic video sequences by appropriately combined di erent content descriptors in a hierarchical framework. Three main modules are involved in the proposed scheme; extraction of reliable depth information, image partition into color and depth regions and a constrained fusion algorithm of color segments using information derived from the depth map. In the rst module, each stereo pair is analyzed and the disparity eld and depth map are estimated. Occlusion detection and compensation are also applied for improving the depth map estimation. In the following phase, color and depth regions are created using a novel complexity-reducing multiresolution implementation of the Recursive Shortest Spanning Tree algorithm (M-RSST). While depth segments provide a coarse representationof the image content, color regions describe very accurately object boundaries. For this reason, in the nal phase, a new segmentation fusion algorithm is employed which projects color segments onto depth segments. Experimental results are presentedwhich exhibit the e ciency of the proposed scheme as content-baseddescriptor, even in case of images with complicated visual content.